Human emulation robot system

ABSTRACT

One aspect of the invention is a robot system comprising a flexible artificial skin operable to be mechanically flexed under the control of a computational system. The system comprises a first set of software instructions operable to receive and process input images to determine that at least one human likely is present. The system comprises a second set of software instructions operable to determine a response to a perceived human presence, whereby the computational system shall output signals corresponding to the response, such that, in at least some instances, the output signals cause the controlled flexing of the artificial skin.

TECHNICAL FIELD OF THE INVENTION

[0001] This invention relates generally to robotics and moreparticularly to a robot and system to emulate human behavior.

BACKGROUND OF THE INVENTION

[0002] The means by which people use a computer may be referred to as aHuman Computer Interface (HCI). An HCI generally comprises outputdevice(s) such as a monitor screen or printer, input device(s) such as akeyboard, touch screen, tablet, scanner, or mouse. Existing robotsgenerally suffer from various disadvantages. For example, existingrobots may not have realistic looking and acting artificial skin. Otherrobots may not be capable of realistically producing appropriate facialexpressions or other nonverbal communications.

SUMMARY OF THE INVENTION

[0003] One aspect of the invention is a robot system comprising aflexible artificial skin operable to be mechanically flexed under thecontrol of a computational system. The system comprises a first set ofsoftware instructions operable to receive and process input images todetermine that at least one human likely is present. The systemcomprises a second set of software instructions operable to determine aresponse to a perceived human presence, whereby the computational systemshall output signals corresponding to the response, such that, in atleast some instances, the output signals cause the controlled flexing ofthe artificial skin.

[0004] The invention has several important technical advantages.Embodiments of the invention may have none, some, or all of thesetechnical advantages without departing from the scope of the invention.The invention provides a more naturalistic mechanism to interact withvarious electronic devices. By providing artificial skin with at leastsome characteristics of human skin, the invention facilitates moreaccurate emulation of human facial expressions and/or gestures. Anotherimportant technical advantage is that the invention may allow perceptionof the emotional state of a human with which a robot is interacting anda simulated emotional response by the robot to better facilitateinteraction between the robot and a human.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] For a more complete understanding of the present invention andthe advantages thereof, reference is now made to the followingdescriptions taken in conjunction with the accompanying drawings inwhich:

[0006]FIG. 1 illustrates one embodiment of a Human Emulation Robot as awhole system.

[0007]FIG. 2 illustrates one process by which an artificial skin may beproduced.

[0008]FIG. 3 illustrates a process by which artificial skin may beproduced.

[0009]FIG. 4 illustrates one process by which one may cause the skin tohave controlled thickness throughout its form.

[0010]FIG. 5 illustrates one embodiment of expression affector, a systemby which expressive movement may be affected in a skin.

[0011]FIG. 6 illustrates one method by which anchors may be embeddedinto the skin by affixing them with pins into the mold of the face-formprior to casting the skin.

[0012]FIG. 7 illustrates one embodiment of a mechanical frame, a systemthat may mechanically support skin.

[0013]FIG. 8 illustrates one embodiment of a system that may be used toaffect naturalistic motion in an artificial lip.

[0014]FIG. 9 illustrates one embodiment of an eye mechanism, a systemthat may be used to affect natural-appearing motion in artificial eyes.

[0015]FIG. 10 illustrates one embodiment of a system that may be used toaffect motion in an elastomeric skin of an artificial human or animal,by a linkage acting within a slot in the mechanical frame.

[0016]FIG. 11 illustrates one embodiment of a system that may be used toaffect the appearance of the dynamic action of the human eyelids.

[0017]FIG. 12 illustrates one embodiment combination of skin, anchors,linkage, actuators, mechanical frame, eye mechanism, eyelids, andassorted sensors, so as to emulate communicative functions of the humanface as a whole.

[0018]FIG. 13 is a block diagram illustrating one embodiment of acontrol system configuration that may be used to emulate the cognitiveaspects of human communication in an HER.

[0019]FIG. 14 is a block diagram illustrating an embodiment of a HumanEmulation Robot being used to access, navigate, or modify theinformation present on a computer network.

[0020]FIG. 15 is a block diagram illustrating a method for producing anartificial skin material.

DETAILED DESCRIPTION OF THE DRAWINGS

[0021] The preferred embodiment of the present invention and itsadvantages are best understood by referring to FIGS. 1 through 15 of thedrawings, like numerals being used for like and corresponding parts ofthe various drawings.

[0022] One aspect of the invention is an HCI system called a HumanEmulation Robot (HER), comprising electromechanical emulation of atleast some naturally-occurring structures, patterns, and/or codesassociated with human communications, in order to generatehuman-computer interactions that may be more innately intelligible andcomforting to humans than previous HCI systems. In one embodiment, anHER comprises apparatus and/or software for an HER to sense and perceiveat least some natural non-verbal human communication signals, apparatusand/or software for an HER to decide meaningful responses to sensoryperceptions, and apparatus and/or software for an HER to emulate, atleast in part, natural human communicative output. To sense and perceivenatural human communication signals, the HER may employ (withoutlimitation) face-tracking machine vision, audio-sensing, facialbiometrics, electronic chemical sensing (smell), and touch sensing.Other hardware and/or software systems may also be used. To decidemeaningful responses to sensory perceptions, the HER may employ any,all, or none of Expert Knowledge Systems, Automatic Speech Recognitionsystems, Natural Language Processing systems, Chatterbots, logicalreasoning systems, and/or statistical reasoning systems. Other relatedsystems may also be used. To emulate human communicative output, the HERmay employ synthesized spoken language and a physically embodied, 3Dmechanical face that is humanlike in appearance and that may display atleast some realistic human aesthetic structures, facial expressions,and/or gestures.

[0023] An HER may be advanced by the coordinated integration of otherdisplay (or other output) technology in addition to saidphysically-embodied, three-dimensional mechanical face. In someembodiments, this additional output technology may supplement thenaturalistic communication with conventional computer graphics and text,sound, etc. Technology that produces various smells may also be used.Technology that produces tactile sensations may also be used. Technologythat produces bodily gestures and/or locomotion may also be used.

[0024] The description below presents many other options that may beassociated with the HER of the invention. The description below alsopresents a number of other structures, methods, compositions of matter,etc., each of which may constitute a separate invention.

[0025] Before describing the drawings in more detail, the followingdescription will provide some additional overview information concerningvarious embodiments and aspects of the invention disclosed herein. Thedescription below discusses various options for various aspects of theHER and its use. Nothing herein is intended to be essential to theinvention or limit the invention in some way not specified in the claimsof any issued patent based upon this description.

[0026] Flexible tissues may be emulated in an HER by using a flexibleelastomeric artificial skin of any suitable substance(s); here andhereafter “skin” shall refer to the material emulation of human oranimal soft tissues. Facial expressions in an HER may be produced by anysuitable means, including but not limited to means described in thispatent. Any suitable actuator technology (such as, for example,servomotor, shape memory alloy, or electroactive polymer (EAP) actuator)may be used anywhere an actuator is described this patent, and anywherean actuator is used in HER systems. All actuators described in use inthis patent may be servomotors, although they are not required to be so.Whenever linkages are required to interconnect skin to an actuator oractuation system, any suitable material such as, for example, cable,thread, braided nylon, kevlar, carbon fiber, or rod may be used; theselinkages shall hereafter be referred to as “linkage”. Such linkage maybe composed of one, two, or more pieces interconnected (such as, forexample, by rotating joints) as desired, which may convert rotational orother motion into linear or other motion as desired.

[0027] The invention may include a series of novel mechanical systemsthat may improve a facial expression mechanism by lowering complexity,cost, weight, and power requirements, such that the system may be mademore effectively mobile, battery-powered, and/or mass-produced. TheHuman Emulation Robot system itself may use one, some, all, or none ofthe described mechanical systems.

[0028] The invention may include a mechanical system that comprises arigid, hollow frame (hereafter called mechanical frame) that is used tosupport various subsystems of a facial-expression display device, suchthat the mechanical frame may serve multiple simultaneous functions. Amechanical frame may be composed of any suitable substantially rigidmaterial, formed into a hollow shell of approximately uniform thickness.The mechanical frame may support multiple structures and systemssimultaneously, which may include skin, facial expression actuators,and/or electronics. The multiplicity of function in a mechanical framemay improve the economy of space, mass, and cost of the manufacturedsystem. Structures of a mechanical frame may emulate aspects of thehuman skull so that when the mechanical frame is covered with anelastomeric artificial-skin that is actuated to affect facialexpressions, the artificial-skin will interact with the frame to form atleast some facial expressions that closely resemble those of humans.

[0029] The invention may include a system for producing a mechanicalframe that is approximately anatomically correct relative to an existingface-form, which may achieve more lifelike expressive deformations inthe artificial skin. This system may utilize forensics data as referencefor skin thicknesses, and notate these thicknesses with depth markersinside the face-form; such face-form may be the interior of a mold of aface-like form, or the interior of a face-like 3D computer-model. Askull-like form may then be modeled based upon the depth of the markers(in inverse proportion) inside the face-form. This represents a use of“facial reconstruction” techniques employed by homicide investigators toreconstruct a face from a skull. Here, the technique is used in reverse,to reconstruct a skull from a face. The resulting structure may also beused as an interior form when molding the skin, so that the skinthicknesses are approximately anatomically correct, and so the skin mayfit well onto the mechanical frame.

[0030] The invention may include a mechanical system, hereafter called“eye mechanism”, that may be used to affect at least somenatural-appearing motion in artificial eyes. In the eye mechanism,artificial-eyes may pivot on points located upon the vertical axes ofsaid eyes. Substantially rigid struts may connect said points to asubstantially rigid frame (hereafter called “eye frame”), such that saidstruts and eye frame together keep the said vertical axes (at leastapproximately) in parallel. Other substantially rigid struts may connectthe eye frame to a hinge mounted on a larger mechanical frame, such thatthe hinge pivots about a horizontal axis that intersects (at leastapproximately) the center of both said artificial-eyes. Thus thevertical axes intersect said horizontal axis at the (at leastapproximate) center of each artificial eye, thus ensuring properball-like rotational behavior of said eyes relative to the mechanicalframe. One or two of any suitable actuators may then affect the rotationof the artificial-eyes about their vertical axes; and any suitableactuator may be used to affect the rotation of the entire eye mechanismaround said horizontal axis of the hinge. To endow a Human EmulationRobot with vision, one may also embed video sensors within theartificial eyes. Video sensors may be excluded or embedded in otherareas of the HER without departing from the scope of the invention.

[0031] The invention may include a mechanical system that endowshuman-appearing mechanical devices and tele-robots with vision withinrealistic-looking eyes. This system may comprise human-appearingartificial eyes, means for affecting movements of said artificial eyes,video sensors embedded within said human-appearing artificial-eyes, andmeans for relaying these images to a controlling processor or a humanoperator.

[0032] The invention may also include a mechanical system that may beused to affect the appearance of the dynamic action of the humaneyelids, and that shall hereafter be called “eyelid mechanism”. Curvedbands or shells of substantially rigid or semi-rigid material (hereafterreferred to as “bands”) may be embedded in the part or parts of the skinthat resembles eyelids. Such bands may be used for upper-lid or forlower lid, or for both. If two bands are used in conjunction to compriseboth upper and lower lids, then as the curved bands follows the forms ofthe lids, the bands may cross near the corners of the eyes, whereat thecurved bands may be attached together with a joint for stabilization.One or both of these joints may also attach to a mechanical frame foradditional stabilization. The portions of the curved bands that thenextend past the corners may continue into the interior space behind theeyes, such that they may operate as levers to be pulled or pushed up ordown by actuators, causing the lids to correspondingly open or close.Alternate means may be used to affect motion in the curved bands, suchas, for example, linkages that attach to the center of the curved bandsand lead to actuators, so that when actuated, the linkages thereby pullor push the lids open and closed.

[0033] The invention may also include a mechanical system formechanically affecting expressions that represent any, all, or none ofthe dynamics of various human facial features and facial expressions,and shall hereafter be called “expression affector”. Expression affectorcomprises a skin made to resemble one or more features of the humanface, flexible anchor(s) that are composed of any suitable material suchas rubber or cloth that is embedded within or otherwise attached to theskin to distribute force through the skin, and linkage(s) that connectan anchor and/or skin to any suitable actuation system. The actuationsystem affects motion via the linkage into the skin, possibly via ananchor, pulling or pushing the skin in a manner that may emulate theactions of the human facial muscles when producing facial expressionsand/or speech. Linkages may connect to anchors by any suitable means,such as, for example, sewing with thread, gluing, and/or hooking with abarbed projection.

[0034] The present invention also relates to several embodiments ofexpression affector, which emulate particular facial actions. Oneembodiment of the expression affector may emulate the appearance of thedynamical pattern of the human smile, affecting action similar to thatof the human Zygomaticus Major muscles, moving the corners of the mouthupwards and outwards. Another embodiment of the expression affector mayemulate the appearance of the dynamical pattern of the human frown,affecting action similar to that of the human Triangularis muscles,pulling the corners of the mouth downward. Another embodiment of theexpression affector may emulate the appearance of the dynamical patternof the human sneer, by affecting action similar to that of the humanCaninus, Levator labii, and Quadratus Labii Superiorus muscles, pullingthe skin upward along the sides of the nose. Another embodiment of theexpression affector may emulate the appearance of the dynamical patternof the human brow-furrow, by affecting action similar to that of thehuman Corrugator tissue, pulling the artificial forehead skin from abovethe eyebrows inward toward the center of the forehead. Anotherembodiment of the expression affector may emulate the appearance of thedynamic action of the human eyebrow, by affecting action similar to thatof the outer portions of the human Frontalis muscle. Another embodimentof the expression affector may emulate the appearance of the dynamicalpattern of the human eye-squint during the action of smiling, byaffecting action similar to that of the human Orbicularis Oculi musclesand the outer portions of the human Quadratus Labii Superiorus muscle,the movements of which cause the portions of cheek just below the outercorners of the eyes to raise up towards the eye. Another embodiment ofthe expression affector may emulate the appearance of certain dynamicalpatterns of the human lower lip caused by the action of the humanDepressor Labii Inferioris muscles.

[0035] The invention may include a mechanical system, hereafter called“sliding affector”, that may be used to affect motion in skin,comprising a substantially rigid linkage that extends from an actuatorthrough a slot that is a gap in the body of a mechanical frame, toattach to the skin, such that the slot guides and channels the motion ofthe linkage, and thus, controls motion of the skin. Inside and outsidethe slot, the linkage may be flanged, so that one flange and perhaps asecond flange may prevent the linkage from slipping through the slot.Such a flanged linkage may be made in two separate flanged pieces thatare inserted from opposing sides of the slot during manufacturing, sothat the pieces adhere together, locking the linkage into the slot. Inone embodiment, this system may be used to emulate the motion of thecenter portion of the human forehead that is caused by the Frontalismuscle, which moves the central brow upward into the forehead, and/orthat is caused by the human Procerus muscle, which pulls the foreheaddown into a scowl.

[0036] The invention may include a mechanical system (hereafter called“lip mechanism”) for affecting with approximate realism, the appearanceof certain dynamical patterns of the human upper or lower lip. Anelastomeric artificial skin-material may be cast into a form thatapproximates a mouth and chin. An anchor, composed of any suitablematerial such as rubber or cloth, may be embedded within the skin of theupper or lower lip to distribute force through the skin, to prevent saidskin-material from tearing, and/or to improve aesthetic performance. Asubstantially rigid linkage, composed of any suitable substance,interconnects said anchor and an actuator in order to affect motion.This linkage may extend vertically down from the lip to the chin foractuating the lower lip (or vertically upwards from the lip into spacebehind the nose for the upper lip), whereat the linkage bends to enterthe interior of the face to attach to the actuator. At this bend, arotating joint may fasten said linkage to a second substantially rigidlinkage, which may be attached or embedded in the artificial skin of thechin for the lower lip. Said second linkage itself may also extend intothe interior of the face to be moved by an actuator, such that, for thelower lip, it may affect an up-down motion within the artificial skin ofthe chin. Because the two linkages are fastened together, said firstlinkage may be made to transmit this up-down motion into the lip (upwardinto the lower lip, or downward into the upper lip). Said first linkage,rotating as a lever about the joint where it meets said second linkage,may also then affect an additional in-and-out motion to the lip whenmoved by another actuator. Alternate means may be used for affecting thein-out motion of the lip without departing from the scope of theinvention, such as, for example, a “bimorph” actuation scheme, whereintwo linkages that are attached together at their ends, extend verticallyinto the lip, such that as one linkage changes in length, it pulls orpushes the second linkage, causing the second linkage to bend. Such abending may push the lip out or pull the lip in. This change in lengthin the linkage may be caused by any other suitable means, such as, forexample: the linkage being pulled or pushed from the interior of thefacial cavity, or material change in dimensions of the linkage (such asmay occur in piezoelectric or EAP materials).

[0037] The invention may emulate the communicative output andcommunicative sensor-input of the human face, in a system knownhereafter as a Human Emulation Display (HED). An HED may, for example,comprise an elastomeric skin, human-appearing artificial eyes, a way tocoordinate facial features into at least some human-emulated facialexpressions and/or speech-like forms (which may include withoutlimitation a smile, a frown, lower lip motion, forehead movement, eyemotion, and jaw motion), a way to visually sense the presence andposition of a person or people, a way to sense human spoken languageand/or a way to transduce synthetic speech into audible signals. An HERand/or a HED may or may not be attached to a mobile body, and may or maynot be endowed with grasping limbs. Alternately, an HER or a HED may beattached to an inanimate body, to no body at all, or to a display-stand.If an HER and/or a HED are attached to an animated artificial body,dynamic gestures in the body, neck, and/or limbs may be used to enhancethe communicative function of the HER and/or the HED.

[0038] Another aspect of the invention that, may, but need not, be usedfor the skin of an HER is a polyurethane elastomer material (hereaftercalled Skin Rubber) that may be advantageous by offering the desirableproperties of both a flexible foam elastomer and a solid elastomer. ThisSkin Rubber comprises a freshly catalyzed Poly(urethane-urea) elastomer(a suitable mixture of a diol and a diisocyanate, that cures into ahigh-molecular weight oligomer, may work, for example “SkinFlex”—aproduct sold by the BJB company) that is introduced by mixing (whilesaid elastomer is still liquid and not yet fully cured) into a freshlycatalyzed Poly(urethane-urea) foaming elastomer (a suitable Polyurethanefoam rubber may work, for example “Flex-foam-it”—a product also sold bythe Smooth-On company) that is also still liquid and curing, theresulting mix curing to become the Skin Rubber polymer material. A ratioof between 15% foam to 85% elastomer, and 50% foam to 50% elastomer maybe used. The resulting Skin Rubber may be a highly stable compositionthat alternates between discrete, microscopic sections of nearly pureelastomer and discrete sections of foaming elastomer. In such instances,the material is easily compressed because it is permeated by gas-bubblevoids, but remains highly elastic because of the action of areas or evennetworks of elastomer.

[0039] Another aspect of the invention may include embodiments of SkinRubber used as an artificial skin, such as may or may not be used inapplications including but not limited to anatomical models,prosthetics, toys, and human-like robotics.

[0040] Skin Rubber may also be used for application embodiments otherthan artificial soft tissues, such as, for example (but withoutlimitation): seat cushions, apparel, mechanical compositions oflocomotion devices (wherein Skin Rubber may be advantageous byfunctioning as a spring and dampener), and various aerospaceapplications.

[0041] Another aspect of the invention that, may, but need not, be usedfor the skin of an HER is an elastomer material composition (hereaftercalled Skin Rubber 2) that may be advantageous by offering the desirableproperties of both a compressible, flexible foam elastomer and a solidelastomer. Skin Rubber 2 may also be used for application embodimentsother than artificial soft tissues, such as, for example (but withoutlimitation): seat cushions, apparel, mechanical compositions oflocomotion devices (wherein Skin Rubber may be advantageous byfunctioning as a spring and dampener), and various aerospaceapplications. Skin Rubber 2 may be achieved by building a matrix ofmaterial that may be removed later in the process. Any suitable materialmay be used to compose this matrix such as, for example: wax, styrene,sugar, or mechanical tooling elements such as, for example: metal rodsthat slide together to interlock in a grid-like mold-matrix. The matrixmaterial may be made into a matrix structure by any suitable means, suchas, for example: automated deposition manufacturing, by stacking asblocks, and/or by self-assembly of structures from shaken, looseparticles. If this matrix is made of separate units of material, theunits may be adhered by any suitable means, such as, for example: bysteeping in solvent or glue, or by partial thermo-melting. Once thematerial matrix is built, any suitable elastomer, such as, for example,room-temperature vulcanizing (RTV) silicone, may be saturated into thematrix and allowed to cure. Once the elastomer is cured, the matrixmaterial can then be removed by any suitable means that may includesolvent-melting, thermo-melting, or bio-consumption (such as, forexample, consumption by microbes or beetles). The material removal maybe assisted by pressurized gas or liquid, or by vacuum. The removal ofthe material matrix may leave voids in place of the matrix, effectivelyresulting in a web of elastomer that may act like a foam, as the voidsare analogous to the gas-bubbles in a flexible foam that allow thecollapse and compression that is characteristic of such foam. In suchinstances, the Skin Rubber 2 may be easily compressible in the manner ofa flexible foam, while maintaining the elastic elongationcharacteristics of the elastomer. Alternately, because the matrix may beprecisely engineered and constructed, structures may be made to existwithin the Skin Rubber 2 that are more elaborate and controlled than thebubbles of a foam. Such controlled structures may include (withoutlimitation): manifolds for controlling the flow of gasses or fluids,folding structures that cause the Skin Rubber 2 to collapse or elongateinto surprising forms under stresses, and structural reinforcements orbuttressing, The structure of the matrix may exist at a variety ofscales, from macroscopic to microscopic and below. The smaller-scalesmay enable many new elastomeric structures for Micro ElectricalMechanical Systems (MEMS), and/or for nanometer scale systems anddevices. Various devices may be embedded into the matrix, so as tobecome subsequently embedded within the Skin Rubber 2; such devices mayinclude (without limitation) sensors, anchors, actuators, and/orelectronics,

[0042] Another aspect of the invention may include embodiments of SkinRubber 2 used as an artificial skin, such as may or may not be used inapplications including but not limited to anatomical models,prosthetics, toys, and human-like robotics.

[0043] Other aspects of the invention may include a number ofapplications of an HER, some of which are described below.

[0044] Various embodiments of an HER may enable a human to usenaturalistic communication to access, navigate, and/or modify theinformation present on computer networks, which may or may not include,but shall not be limited to, the following: the Internet, the World WideWeb (WWW), Wi-Fi, P2P (peer to peer), and/or Grid computing networks.Furthermore, embodiments of an HER may be used to access, navigate,and/or modify the information present on computers, computationalprocessors and/or information storage devices. Embodiments of an HER maybe used to display the condition of a computer and/or as a replacementfor error codes. Embodiments of an HER may be used as an automatedassistant to humans, helping to organize and manage human affairs.

[0045] Embodiments of an HER may be used to automate customer servicefunctions, which may or may not include without being limited to:drive-through service at fast food restaurants and banks, informationkiosks, automatic teller machines, restaurant service in general, salesagents in retail stores, ticketing agents for travel or sports, andsales agents in general. In each of these cases, the HER may be used forone, some, all, or none of the following: (a) to obtain information fromthe customer as to desired goods or services to purchase, (b) to obtaininformation from the customer as to what they are purchasing forpurposes of calculating the amount of money owed by the customer, (c)obtaining information from the customer as to their method of paymentand other relevant payment information (such as, for example, an accountnumber), (d) a good or service for which a customer requiresinformation, (e) obtaining and/or verifying a password or PIN associatedwith particular goods or services, (f) communicating with the customerconcerning any of the foregoing, and (g) performing any other servicesthat may be automated that are currently performed by humans or machinesacting in any of the listed roles. Embodiments of an HER may be used asa commercial advertisement display (e.g. as a clothes mannequin), suchthat the said display may engage a person or people in attentivedialogue, and answer direct questions about the goods or services beingadvertised. Thus, the HER might also act like an automated sales ormarketing representative who seeks to educate consumers and/or sellproducts to consumers in grocery stores, department stores, malls, otherpublic places, etc. Embodiments of an HER may be used as a promotionalor informative display, such that the display may engage a person orpeople in attentive dialogue about a select subject, and answer directquestions about the topic being promoted or information beingdisseminated.

[0046] Embodiments of an HER may be used as a therapy tool, such thathuman interaction with said device may have healthful physical,neurological, and/or psychological effects. Embodiments could also beused for automated psychological therapy or as an interactiveneurological therapy tool for individuals with social communicationdisorders. In each of these cases, the HER may be used for one, some,all, or none of the following: (a) to obtain data from the patientregarding patient condition by natural language dialogue, by accessingpatient records, by olfactory sensing, tactile sensing, and/or by visualinspection of the patient, (b) to use dialogue with the patient tomaintain the attention of the patient, and to elicit information fromthe patient, (c) to use expert systems database(s) (that may or may notbe based on standardized psychology knowledge) and/or various reasoningsystems to perform a diagnosis of the patient based on the obtaineddata, (d) to use naturalistic display and verbalizations to delivertherapeutic interactions, utterances, gestures, and prescriptions forbehavior to the patient, (e) to use naturalistic dialogue to deliverassessments of a patient to a human, such as a health-care worker, (f)to interactively train a patient's communicative abilities and relatedcognitive functions, (g) to perform any other services that may beautomated that are currently performed by humans or machines acting inany of the listed roles.

[0047] Embodiments of an HER may be used for loneliness alleviation forindividuals with limited social contact, such that sociable stimulationmay provide healthful benefits. In this case, the HER provides socialand/or informative interaction to an individual, and may relay messagesto/from healthcare workers and/or family, via a computer network,telephone, or other telecommunication means. Embodiments might be usedas a stress reduction device, such that human interaction with saiddevice may comfort a person or ameliorate troubled mental states. Inthis case, the HER may be used to produce soothing language and gesturesto calm the person, in response to detected language and/or behavior ofa patient, wherein such response may be decided by an expert systemdatabase that may or may not be based upon standard psychology.

[0048] Embodiments of an HER may be used as an automated educator,mentor, or tutor. In these cases, the HER may be used for one, some,all, or none of the following: (a) to provide nurturing attention to thepupil, (b) to maintain the attention of the pupil, by varied means thatmay include games, linguistic and paralinguistic interaction, and/orstorytelling, (c) to obtain data from the pupil by natural languagedialogue, visual inspection, olfactory (chemical) sensing, and/oraccessing records of the pupil, (d) to perform diagnosis and assessmentof the pupil, by processing obtained data using expert systemsdatabase(s) and reasoning systems that may or may not be based uponstandard knowledge of pedagogy and psychology, (e) to use naturalisticdisplay and verbalizations to deliver knowledge, training,encouragement, assignments, and/or commands/suggestions for behaviorchange to the pupil, (f) to provide to human educators or parents,assessments of a pupil's behavior, performance, and progress (g) toperform any other services that may be automated that are currentlyperformed by humans or machines acting in the listed roles.

[0049] Embodiments of an HER may be used as a developmental nurturingtool such as an educational play object, a developmentally-nurturingplay device, a developmental therapy device, an automated nanny, and/oran automated developmental therapist, such that sociable interactionwith said nurturing tool may provide healthful developmental benefits tochildren. In these cases, the HER may be used for one, some, all or noneof the following: (a) to provide nurturing attention to the child, (b)to maintain the attention of the child, by varied means that may includegames and/or storytelling, (c) to obtain data from the child by naturallanguage dialogue, visual inspection, tactile sensing, olfactory(chemical) sensing, and/or accessing records of the child, (d) toperform diagnosis and assessment of the child's data, by using expertsystems database(s) and reasoning systems that may or may not be basedupon standard knowledge of pedagogy and developmental psychology, (e) touse naturalistic display and verbalizations to deliver knowledge,encouragement, assignments, and/or commands/suggestions for behaviorchanges to the child, to therapeutically influence the child withinteractions, utterances, and/or gestures, and/or to socialize thechild, (f) to provide assessments of a child's behavior, developmentalperformance, and progress to parents, medical personnel, and/oreducators (g) to perform any other services that may be automated thatare currently performed by humans or machines acting in the listedroles.

[0050] Embodiments of an HER may be used as toys such as dolls,novelties, stuffed animals, and/or display devices for video games. Inthese cases, the HER may be used for one, some, all or none of thefollowing: (a) to maintain the attention of the child, by varied meansthat may include games, interactive dialogue, and/or storytelling, (c)to obtain data from the child by natural language dialogue, visualinspection, tactile sensing, olfactory (chemical) sensing, and/oraccessing records of the child, (d) to perform diagnosis and assessmentof the child's data, by using expert systems database(s) and reasoningsystems that may or may not be based upon the knowledge of standardpedagogy and developmental psychology, (e) to use naturalistic displayand verbalizations to deliver entertainment and/or knowledge to thechild, (f) to provide assessments of a child's behavior, development,and/or progress, to parents, and/or educators (g) to connect into acomputer network for multi-user games, and/or to access additionalknowledge and/or data for enhanced playful function.

[0051] Embodiments of an HER may be used as an artistic medium includingwithout limitation automated sculpture, automated actors, interactiveautomated theater, and/or automated festivity spaces. In these cases,the HER may be used for one, some, all or none of the following: (a) toprovide aesthetic, cultural, and/or narrative expression for an artistor artists, (b) to maintain the attention of the viewer by varied meansthat may include games, linguistic interaction, performance, automatedenvironmental changes, and/or storytelling, (c) to obtain data from aperson or people by natural language dialogue, visual inspection,olfactory (chemical) sensing, and/or accessing records, (d) to performassessment of said data by using expert systems database(s) and/orreasoning systems that may or may not be based upon the knowledge ofgame theory, psychology, narrative arts, narrative theory, art theory,aesthetic theory, theatrical design and legerdemain, (e) to serve as aninteractive portrait medium, (f) to use naturalistic display,verbalizations, and automated environmental changes, to deliver sensoryimpressions to the viewer, with the objective of cultural orpsychoactive effect, (f) to provide recordings and assessments ofpeople's response to the artwork for a supplement to the artwork, forhistorical posterity, and/or for scientific evaluations, (g) to performany other services that may be automated that are currently performed byhumans or machines acting in the listed roles and art forms.

[0052] Embodiments of an HER may be used as interactive automatedmemorial icons, to memorialize deceased and/or celebrated individuals.In such embodiments, the HER may be used for one, some, all or none ofthe following: (a) to emulate the appearance of an individual, (b) toemulate the personality of an individual, such as may or may not becharacterized by speech patterns, specific gestures, idiosyncraticphrasings, and/or any other distinguishing states or behaviors, (c) toenact words and stories of the portrayed individual, as may be obtainedfrom writings, video recordings, audio recordings, or any other suitablemeans, (d) to recognize family and friends of the individual by sightand/or by audio, and to greet persons by name and/or with eye contact,(e) to hold personalized interactions with family and friends,recounting stories and events specific to the person(s) with whom theHER interacts, (f) to portray a benevolent entity, such as, for example,an angel, that may offer solace to family and friends visiting thememorial.

[0053] Embodiments of an HER may be used to search for missing children,by employing biometric identification software to analyze HER videodata. In this case, the HER may be used for one, some, all or none ofthe following: (a) to captivate a child's attention with an entertaininginteraction or performance, so that direct eye-to-camera gaze isobtained from the child (b) to compare captured video images of a childto databased images of missing children, (c) to notify authorities uponthe identification of a potential missing child, (d) to use engaginginteraction to maintain the attention of the child while authorities arein transit to the site.

[0054] Embodiments of an HER may be used to search for wanted individualsuch as at security checkpoints, by employing biometric identificationsoftware to analyze HER video data. In this case, the HER may be usedfor one, some, all or none of the following: (a) to captivate asubject's attention with an engaging, authoritative interaction, so thatdirect eye-to-camera gaze is obtained from the subject (b) to comparecaptured video images of a subject to databased images of wantedindividuals, (c) to notify authorities upon the identification of apotential wanted individual.

[0055] Embodiments of an HER may be used as a tele-presence device, suchthat said device displays expressive behavior that is being controlledby a person from some distance away, displaying said expressive behaviorto one or more persons. In another tele-presence application, the HERcould be used to perform medical examinations on a patient who isisolated or in quarantine, delivering the sense of face-to-faceinteractions without compromising the quarantine. In these cases, theHER may be used for one, some, all or none of the following: (a) toprovide the comforting impression of the three-dimensional presence of aperson, (b) to communicate across some distance the expressions ofseparated parties, (c) to perform any other functions that may bepartially automated that are currently performed by humans inface-to-face encounters.

[0056] Embodiments of an HER may be used as an automated MedicalDiagnosis tool, for obtaining medically relevant information frompatient using emulated natural social interaction with patient. In thesecases, the HER may be used for one, some, all or none of the following:(a) to provide nurturing attention to the patient, and the soothingimpression of the presence of a person, (b) to obtain data from theperson by natural language dialogue, auditory inspection, visualinspection, olfactory (chemical) sensing, tactile sensing, temperaturesensing, and/or accessing patient records, (c) to perform diagnosis andassessment of said data, by using expert systems database(s) and/orvarious reasoning systems that may be based upon the standardaccumulated knowledge of medicine, (d) to use naturalistic display andverbalizations to deliver to the patient knowledge, diagnosis,prognosis, prescriptions/advice for behaviors, and/or other utterancesor gestures to therapeutically influence the patient, (e) to recordassessments of a patient's condition, interaction, and/or progress (f)to communicate assessments regarding a patient's condition and/orprogress to a medical care worker or professional, by naturalisticdialogue, (g) to perform any other services that may be automated thatare currently performed by humans or machines acting in healthcareroles.

[0057] Embodiments of an HER may be used as a model for education and/orreference, which may include without limit to: interactive anatomicalmodels, interactive medical reference models and/or models fordemonstrating principles of psychology, social sciences, and/orcognitive science. In these cases, the HER may be used for one, some,all or none of the following: (a) to simulate various human-likeanatomical actions for the purposes of demonstrating human physiology,anatomy, aesthetics, human-factors, and/or psychology, (b) to simulatenatural interactive dialogue for the purpose of demonstrating principlesof psychology, neuroscience, cognitive science, sociology, humanrelations, and/or other sciences and/or disciplines, (c) to introduce,describe, and guide the user academically through the reference oreducational exercise, (d) to transition the trainee or student into theeducational scenario, by natural dialogue interaction, (e) to act out arole in a theatrical manner appropriate to the educational or referencescenario, such as, for example, an HER acting as an aphasic patient maypretend to not recognize certain words or thoughts in a dialogue, inorder to demonstrate the condition of partial aphasia, (f) to perceiveand respond to the action of the user in a theatrical manner thatenhances the reference education, (g) to quiz the user, or to providefeedback on user performance during educational exercises, (h) toperform any other services that may be automated that are currentlyperformed by humans or machines acting in listed roles.

[0058] Embodiments of an HER may be used as a dummy for training andsimulation, which may include without limit to: animated CPR trainingdummies, devices for surgical simulation, interactive simulation devicesfor police training, simulation devices for customer service training,sales training, and/or military training. In these cases, the HER may beused for one, some, all or none of the following: (a) to simulatevarious human-like movements for the purposes of making trainingexercises more realistic, (b) to engage the trainee with eye-to-eyecontact, gestural interaction, and naturalistic dialogue in a mannerpertinent to the topic of the training, in order to make the trainingmore realistic, (c) to obtain data on the trainee by verbal interaction,by visual sensing, by tactile sensing, by temperature sensing, byolfactory (chemical) sensing, and/or by accessing records, (d) to assessthe trainee using expert systems database(s) and/or reasoning systemsthat may be based upon the standard training in field of use, (e) tocustomize the training or simulation in response to data obtained aboutthe trainee, (f) to introduce, describe, and guide the traineeacademically through the training, reference or educational exercise,(g) to transition the trainee into the educational scenario, by naturaldialogue interaction, (h) to act out a role in a theatrical mannerappropriate to the training scenario, such as, for example, an HERacting as a CPR dummy may pretend to choke severely in order to enhancethe realism of the CPR training exercise, (i) to perceive and respond tothe action of the trainee in a theatrical manner that enhances thetraining or education, (j) to provide the trainee feedback onperformance during the training exercise, (k) to record and reportassessments regarding a trainee's performance, (l) to perform any otherservices that may be automated that are currently performed by humans ormachines acting in listed roles.

[0059] Embodiments of an HER may be used for testing and/or developingnew computer software in general, and sociable Artificial Intelligence(AI) software in particular. In these cases, the HER may be used forone, some, all or none of the following: (a) to provide human-likefacial expression, sensing, and communications to software, as means forthe software to interact more naturally with humans (b) to allowsoftware to obtain data from users by natural language dialogue, visualinspection, olfactory (chemical) sensing, and/or accessing records, sothat the software may be more effective at interacting with humans, (c)to allow software to perceive and converse with a person so as tocustom-adjust to the preferences of an individual human, (c) to providesoftware developers the ability to use natural communication techniquesto give to software feedback on its performance, (d) to allow softwareto communicate its condition naturalistically to human softwaredevelopers, (e) to serve as a nexus for integrating various softwaresystems into a more capable whole system.

[0060] Embodiments of an HER may be used as a platform for testing andmodeling scientific theories and hypotheses, which may or may not beused within the following fields: Neuroscience, Cognitive Science,Psychology, Sociology, Linguistics, Paralinguistics, Human FactorsEngineering, and electroactive polymer (EAP) science. In these cases,the HER may be used for one, some, all or none of the following: (a) toprovide means for representing theorized systems of human and animalcommunication, cognition, and society as means of empirically evaluatingtheories of such systems, (b) to provide a test platform for newmaterials and material science, for sensor science and technology, andfor sciences of new and emerging actuators including electroactivepolymer (EAP) actuators, (c) to serve as a platform for testing anddeveloping control theory and control systems, (d) to provide a testplatform for the science and technology of the mechanics of simulatedhuman communications (e) to provide controlled communicative stimulus toliving subjects to study the response of the subjects, (f) to assistwith the design/engineering of systems and/or objects that need to beused by humans, by serving as a surrogate person when testing the systemor object, (g) to assist with the design/engineering of systems and/orobjects that emulate human communicative expression and/or cognition(such as, for example, HER systems themselves), (h) to automate theaccumulation of data of human communications, cognition, and/orsociology for scientific assessment, by obtaining data from human testsubjects via natural language dialogue, visual inspection, olfactory(chemical) sensing, tactile sensing, temperature sensing, and/oraccessing records, (i) to assess collected data with software such asautomated perception systems, statistical evaluation software, and/orartificial intelligence, (j) to use naturalistic dialogue to helpcommunicate data and/or scientific evaluations to people.

[0061] Embodiments of an HER may be used to automate or to enhancepolicing and security activities. In these cases, the HER may also beused for one, some, all, or none of the following: (a) to simulateconvincingly an officer or security guard, by making and maintaining eyecontact with a person who enters the vicinity, and affectedlyscrutinizing the person as if the person is suspicious, so as to cause apotential criminal to be reluctant to commit crime, (b) to interact in afriendly manner with people that are known and identified, (c) to obtaindata by visual inspection, natural language dialogue, olfactory(chemical) sensing, temperature sensing, and/or accessing records, (d)to assess said data (including but not limited to nonverbal expressionand language) with reasoning systems and/or expert systems database(s),to determine possible security threat, conflict and/or crisis situation,to automatically identify known criminals, or wanted and/or dangerousindividuals (e) to employ an expert system and/or reasoning system toguide HER behavior, including interaction with humans in a crisis, (f)to interact with the public in a manner that is generally friendly andbuilds community trust, (g) to operate as a security or fire alarm,sounding an alarm upon a detected danger, and/or notifying outsideagencies, such as police or fire departments or a security company, (h)to record audio, video, testimony, and automated assessments of data,(i) to coordinate and communicate with a larger police or securitynetwork, (j) to communicate by naturalistic dialogue, data andassessment of security activities, to authorized personnel, (k) toperform any other services that may be automated that are currentlyperformed by humans or machines acting in policing or security roles.

[0062] Embodiments of an HER may be used to provide a sociable interfacefor military machines, which may be advantageous by enabling fastercommunication from machine to human, such as is inherent to innatehuman-to-human communication, thereby expediting time-sensitive fieldoperations. In these cases, the HER may be used for one, some, all, ornone of the following: (a) to provide means for faster coordination ofhuman and machine, particularly in battle situations, (b) to obtain datafrom a soldier, from the field, from civilians, and from enemycombatants and associated paraphernalia of war by visual inspection,olfactory (chemical) sensing, natural language dialogue, tactilesensing, temperature sensing, and/or accessing records, (c) to performdiagnosis and assessment of said data, by using expert systemsdatabase(s) and/or reasoning systems, (d) to deliver to a soldier byusing naturalistic display and verbalizations, data, data assessments,and/or orders transmitted from commanding officers, (e) tonaturalistically interact with civilians and possible enemy combatants,in order to disambiguate civilians from combatants, to obtain data, andto function as sentinels and as guards at checkpoints, (f) todisseminate to civilians and combatants propaganda and information, (g)to interact with injured soldiers as a medic, obtaining data fordiagnosis by naturalistic dialogue, and to deliver the comfortingsensation of the emulated presence of a caring human being, (h) toperform any other services that may be automated that are currentlyperformed by humans or machines acting in military roles.

[0063] Embodiments of an HER may be used as a rescue robot enhancement,which may be advantageous by improving interaction with found survivorsand may deliver to a survivor the comforting sensation of the emulatedpresence of another human being. In these cases, the HER may be used forone, some, all, or none of the following: (a) to provide nurturingattention to the survivor, and the soothing impression of the presenceof a person, (b) to obtain data from the person by natural languagedialogue, visual inspection, olfactory (chemical) sensing, tactilesensing, temperature sensing, and/or accessing patient records, (c) toperform diagnosis and assessment of said data, by using expert systemsdatabase(s) and reasoning systems that may or may not be based upon thestandardized accumulated knowledge of medicine, (d) to use naturalisticdisplay and verbalizations to deliver messages to the survivor, and/orto therapeutically influence the survivor, (e) to record and relayassessments of a survivor's condition, interaction, and/or progress (f)to communicate to a medical care worker or professional, by naturalisticdialogue or other means, assessments regarding a survivor's conditionand/or progress, (g) to perform any other services that may be automatedthat are currently performed by humans or machines acting in the rolesof medic or paramedic.

[0064] While the above description has provided various options forimplementing an HER and various applications of an HER, the describedHER may omit some or all of the described mechanical, electronic,electromechanical, software, and other systems without departing fromthe scope of the invention. Other systems could be substituted for thosedisclosed. While certain materials have been described, other materialscould be used without departing from the scope of the invention. Whilevarious applications of an HER have been described, the HER could beused for other applications without departing from the scope of theinvention. While an HER could have a capability to perform all of theapplications described herein, it might also perform none, some, orother functions without departing from the scope of the invention. Whilecertain materials are described herein as desirable, other materials maybe used without departing from the scope of the invention. It willfurther be understood by those skilled in the art that descriptions ofmechanical systems above may include electronic, electromechanical, andbiological components. Such systems may involve computer software aswell.

[0065]FIG. 1 illustrates one embodiment of a Human Emulation Robotsystem, including an HED 12, and an electronic control system 13 thatgoverns the operation of various mechanisms in order to emulate at leastsome verbal and nonverbal human communications. HED may include videosensors 60, audio sensors 67, skin 25, anchors 34, linkages 35, and anaudio transducer 70. Data may be sent from the HED sensors to a computerby any suitable communications medium, including without limitation awireless link, while control signals for speech and motor control may bebrought into the embodiment by any suitable communications medium,including without limitation a wireless link. The same or separatecommunication link(s) could be used for both inputs and outputs andmultiple communication links could be used without departing from thescope of the invention. Expressive functions of the face may be achievedusing anchors 34, linkages 35 and actuators 33, organized to emulate atleast some natural muscle effect. Sensor data may be relayed into acomputational system 88, which in the figure comprises a computer andvarious software, but could exist within microcontroller(s), a computernetwork, or any other computational hardware and/or software. Thefunctions performed by computational system 88 could also be performedin whole or in part by special purpose hardware. Although thecomputational system 88 is portrayed in FIG. 1 as existing externally tothe HED, alternatively the computational system 88 may be partially orentirely enclosed within the HED without departing from the scope of theinvention. Automatic Speech Recognition (ASR) 89, may process audio datato detect speech and extracts words and low-level linguistic meaning.Computer Vision 90 may perform any of various visual perception tasksusing the video data, such as, for example, the detection of humanemotion. Decision processes 91 may determine appropriate communicativeresponses and other behaviors based upon the input received. SpecialPrograms 92 may be used to enhance the functionality of the system.Specific Programs 93 is other software that is accessed by the HER. Datastorage 94 affects long term memory, builds knowledge, and storessoftware. Speech responses may be produced by any suitable speechsynthesis software 95 and made audible using an audio transducer 70. Anysuitable motor controller 96 may be used to control actuators toeffectuate non-verbal expressions in the HED 12. Some or all of thecomponents of computational system 88 could be omitted or othercomponents added without departing from the scope of the invention. Withrespect to the software modules, FIG. 13 shows more specifically whateach is doing, and describes commercially available, off-the-shelfoptions for each. While in the described embodiments, sensors aredescribed as being placed within the HER emulated human face, HERdevices do not require any of the sensing elements (which may include,without limit to, video cameras, microphones, olfactory sensors, touchsensors, velocity sensors, accelerometers, and/or positioning sensors)to be integral to the HED. Any and/or all sensors may be placed in otherlocations external to the HED (or in other locations within the HED thanare described herein) without changing the scope of the invention.Additionally, other sensors may or may not be utilized as desired, suchas, for example, bar-code scanners, infrared sensors, ultra-soundsensors, echo-location sensors, touch sensors, velocity sensors,accelerometers, positioning sensors, fingerprint scanners, retinalscanners, sphygmomanometers, pulse detectors, EEG detectors, terahertzimaging devices, magnetic resonance imaging devices, ground penetratingradar, skin current detectors, and/or any suitable other sensing,perception, or data-gathering and analysis instrumentation andtechnologies. An HER may or may not also access data from externalsensors, potentially including sensors contained within other devices(such as, for example, weather stations, automobile sensors, “smartdust” sensors, other robots, and/or other HER devices), via any suitablemeans of data transmission, such as, for example, a computationalnetwork.

[0066]FIG. 2 illustrates one process by which an artificial skin 25 maybe produced. In the first step of the process, a human-like face form 24is obtained by any suitable means that may or may not include sculpting,life-cast, or computer modeling and may be produced using ComputerNumerical Control (CNC) milling. To then transfer said face-form into asubstantially supple elastomer skin, one may make a mold 26 (negativeimpression) of said face-form. Such a mold may be made using a varietyof known materials and a variety of known methods. Into this mold, anysuitable elastomer may be poured, preferably while still liquid andcatalyzing, and then said elastomer may be allowed to cure so that saidelastomer adopts the form of said human-like face-form. Any suitableelastomer may be used for the skin, such as Thermoplastic, Silicone,Polyvinyl, Polyethylene, Polyurethane, Polyolefin, or the elastomersdescribed herein as “Skin Rubber” and “Skin Rubber 2”. If desired, theinterior of the skin may be made hollow by any suitable process such as,for example, the process shown in FIG. 4. Additionally, if desired,hollow sockets may be made in the skin to accommodate artificial eyes,by affixing eyeball-shapes within the mold 26 before casting the skin,so that the eyeball-shapes impress hollow sockets into the skin.

[0067]FIG. 3 illustrates a process by which Skin Rubber may be produced.Skin Rubber may be made from a freshly catalyzed Poly(urethane-urea)elastomer 27 (any suitable mixture of a diol and a diisocyanate, thatcures into a high-molecular weight oligomer, might be used, such as, forexample “Evergreen 10”—a product sold by the Smooth-On company) that isintroduced by mixing (while said elastomer is still liquid and not yetfully cured) into a freshly catalyzed Poly(urethane-urea) foamingelastomer 28 (any suitable Polyurethane foam rubber may work, such as,for example “Flex-foam-it”—a product also sold by the Smooth-On company)that is also still liquid and curing. The resulting material cures tobecome Skin Rubber 29. A ratio of between 15% foam to 85% elastomer and50% foam to 50% elastomer may be used.

[0068]FIG. 4 illustrates one process by which one may cause the skin tohave controlled thickness where desirable or throughout its form, sothat the skin may be made to deform into expressions in a naturalisticmanner. In this process, the inner structure of the skin may be formedusing an inner mold 30 that registers within the outer mold 26, suchthat the skin is then cast between the two. The interior structure ofthe resulting skin 32 may also correspond to the structure of a rigidmechanical frame, so that the skin may fit nicely onto a supportingmechanical frame.

[0069]FIG. 5 illustrates one embodiment of an expression affector, amethod by which expressive movement may be affected in a skin 25 by anactuator 33 pushing or pulling on an anchor 34 that is embedded withinthe skin 25. The anchor 34 and actuator 33 may be interconnected bylinkage 35. In this embodiment, actuator 33 and the skin 25 are bothmounted on a substantially rigid mechanical frame 36. Actuator 33 may beconnected to mechanical frame 36 by a mounting bracket 37. As actuator33 pushes or pulls the linkage 35, force is transmitted to the anchor34, which then distributes the force through a region of the skin 25.The linkage 35 may pass through a Gap 38 in the mechanical frame 36, sothat the actuator can be mounted on the interior of the mechanical frame36. The mechanical frame may be used to serve multiple simultaneousfunctions, which may improve the economy of space and weight bysupporting multiple structures simultaneously, wherein said structuresmay or may not include, without being limited to: facial expressionactuators 33, supporting brackets 37, and/or artificial skin 25. In theillustrated embodiment, the various described elements may operatetogether to produce a smile, simulating the action of the ZygomaticusMajor muscle. However, this mechanism may also be used in other parts ofthe face to emulate the actions of other facial muscles, as is depictedin FIG. 12. It may also be used to emulate the appearance of actions ofmuscles in the body. Actuator 33 comprises a servomotor, but may bereplaced by any suitable actuator (such as, for example, shape memoryalloy or electroactive polymer actuators).

[0070] While actuator 33 in this embodiment both pushes and pullslinkage 35, actuators could be used where suitable for either pushing orpulling standing alone. Moreover, a different actuator could be used forpushing than for pulling linkage 35. The illustrated actuators,linkages, and anchors are only examples, and other ways of moving skin25 could be used without departing from the scope of the invention.

[0071]FIG. 6 illustrates one method by which anchors 34 may be embeddedinto the skin 25 by affixing them with pins 39 into the mold 26 of theface-form prior to casting the skin. When the liquid elastomer 40 of theskin is poured over the anchors 34, the anchors 34 become secured withinthe skin as the elastomer cures into a skin 25. FIG. 6b shows theresulting skin 25 with embedded anchors 34. Other forms of coupling anactuator (directly or indirectly) to the skin 25 to achieve movementthereof could be used without departing from the scope of the invention.

[0072]FIG. 7 illustrates one embodiment of a mechanical frame 36 thatmay mechanically support the skin 25. This mechanical frame 36 may beskull-like in form to enable the artificial-skin 25 to ride over theframe into expressions that, in at least some cases, closely resemblethose of the natural human face. Skin 25 may be attached to mechanicalframe 36 by any suitable means.

[0073]FIG. 8 illustrates one embodiment of a system (lip mechanism) thatmay be used to affect naturalistic motion in an artificial lip 45. Anelastomeric artificial skin 25 is cast into a form that approximates alower lip 45 and a chin 46. A substantially flexible anchor 34, such as,for example, one made of rubber or cloth, may be embedded within theskin of the lower-lip 45 to distribute force through the skin 25, toprevent the skin from tearing and to improve aesthetic performance. Asubstantially rigid or semi-rigid linkage 47, composed of any suitablesubstance, may connects said anchor 34 to an actuator 48 in order toaffect motion. This linkage 47 may extend vertically down from the lipinto the chin 46, whereat the linkage may bend to enter the interior ofthe face to be actuated. At this bend, a rotating joint 49 may fastensaid linkage to second substantially rigid linkage 50, which is embeddedin the artificial skin of the chin 46. Said second linkage 50 may alsoextend into the interior of the face to be moved by actuator 51, suchthat it affects an up-down motion within the artificial skin of the chin46. Because the linkages (47 and 50) are fastened together in thisembodiment, the first linkage 47 is responsive to the motion of thesecond linkage 50 to transmit this up-down motion upwards into the lowerlip 45. First linkage 47, rotating as a lever about the joint 49 whereit meets said second linkage 50, also then affects an additionalin-and-out motion to the lower lip when moved by another actuator 48.All actuators may be substantially rigidly connected to a mechanicalframe 36, by a mounting bracket 37.

[0074]FIG. 9 illustrates one embodiment of an eye mechanism that may beused to affect at least some natural-appearing motion in artificialeyes. Artificial-eyes 52 pivot on points 53 located upon the verticalaxes of said eyes. Substantially rigid struts 54 connect said points toa substantially rigid frame 55 (hereafter called “eye frame”), such thatsaid struts and said frame together keep the said vertical axes (atleast approximately) in parallel. A hinge 56 may connect said frame to alarger mechanical frame 36, such that the hinge 56 pivots about ahorizontal axis that intersects the (at least approximate) center ofboth said artificial-eyes 52. Substantially rigid strut or struts 57reach from said hinge 56 to connect to the eye frame 55, with geometricproportions such that each of said vertical axes intersect saidhorizontal axis at the (at least approximate) center of each artificialeye. This geometry promotes proper ball-like rotational behavior of saideyes relative to mechanical frame 36. Two actuators 58 may then affectthe rotation of the artificial-eyes 52 about their vertical axes; andone more actuator 59 may affect the rotation of the entire eye mechanismaround said horizontal axis of the hinge 56. To endow a Human EmulationRobot with vision, one may embed video sensors 60 within the artificialeyes 52.

[0075]FIG. 10 illustrates one embodiment of a system, sliding affector,that may be used to affect motion in skin 25. This system comprises asubstantially rigid linkage 35 that extends from an actuator 33 througha slot 61 in the body of a mechanical frame 36 to attach to an anchor 34in the skin 25, such that the slot 61 guides and channels the motion oflinkage 35 when the linkage is moved by actuator 33, thus controllingmotion of the skin 25. Inside and outside the slot 61, the linkage 35may be flanged, so that one or two flanges 62 may prevent the linkage 35from slipping through the slot 61. Such a flanged linkage may be made intwo separate pieces that are inserted into opposite sides of slot 61during manufacturing, so that they adhere together, effectively lockingthe linkage 35 into the slot 61. Any number of pieces could be used,however, without departing from the scope of the invention. In thedepicted embodiment, this system may be used to emulate the motion ofthe center portion of a human forehead, caused by the Frontalis muscle,which moves the central brow upward into the forehead, and/or by thehuman Procerus muscle, which pulls the forehead down into a scowl.

[0076]FIG. 11 illustrates one embodiment of an eyelid mechanism systemthat may be used to affect the appearance of the dynamic action of thehuman eyelids. Curved bands 63 of substantially rigid or semi-rigidmaterial may be embedded in the part or parts of the skin 25 thatresemble eyelids 64. Such bands may be used for the upper-lid, the lowerlid, or both. If two bands are used in conjunction to comprise bothupper and lower lids, then as the curved bands follow the forms of thelids the bands may cross, whereat the curved bands may be attachedtogether as a joint 65 for stabilization. One or both of these joints 65may be additionally attached to a mechanical frame 36 for additionalstabilization. The portions of the curved bands 63 that then extend pastthe corners may continue into the interior space behind the eyes, suchthat they may operate as levers 66 to be pulled or pushed up or down byactuators 33, causing the lids 64 to correspondingly open or close.Optionally, the curved bands 63 may be embedded within the skin of theeyelids by gluing, or by placement in the skin mold prior to casting theskin, in like manner used to embed anchors in FIG. 6. Alternatively, anyother suitable means may be used to move the eyelids of an HER, or saideyelids may not be moved independently at all, without departing fromthe scope of the invention.

[0077]FIG. 12 illustrates one embodiment of an HED, wherein skin 25,anchors 34, linkages 35, actuators 33, mechanical frame 36, eyemechanism 9, eyelid mechanism 11, and sensors 60, 67, 68, are configuredso as to emulate communicative functions of the human face as a whole.Sensors distributed through the HED may include video sensors 60,embedded in the artificial eyes 52, audio sensors 67 in the artificialears 69, and touch sensors 68 embedded in the artificial skin 25.Additionally, an audio transducer 70 may be present to project syntheticspeech. In this embodiment, data may be sent to and from the HED by acable 71. Alternatively, any of the communication link options discussedabove could also be employed with this embodiment. Alternatively, all orpartial computational apparatus used to achieve HER control processesmay be contained within the HED without departing from the scope of theinvention. Expressive visual-display functions of the face may beachieved by eye mechanism 9, eyelid mechanisms 11, and anchors 34,linkages 35 and actuators 33, which may be assembled into expressionaffectors 5 and/or sliding affectors 10. Alternatively, any other meansmay be used to emulate human expressive visual-display without departingfrom the scope of the invention. As in the illustrated embodiment, anyor all of the major muscle groups present in the natural anatomy of thehuman face 72 may be represented and emulated. These major muscle groupsmay include Frontalis 73, Corrugator 74, Orbicularis oculi 75, Procerus76, Levator labii superioris 77 (when emulated, the effect of thismuscle may or may not be treated as interchangeable with that ofCaninus), Zygomaticus major 78, Zygomaticus minor 79, Buccinator 80(when emulated, the effect of this muscle may be treated asinterchangeable with that of Risorius), Depressor labii 81, Orbicularisoris 82, Mentalis 83, Triangularis 84 (when emulated, the effect of thismuscle may be treated as interchangeable with that of Depressor angulioris), Nasalis 85, and Superior Tarsal Muscle 86 (which generallyoperates in conjunction with and similarly to the Levator palpebrae andthe palpebral muscles of the Orbicularis oculi). Additionally, othermuscles may or may not be emulated, such as, for example, those thatachieve jaw movements-Jaw muscles 87, and/or those that achieve head andneck poses. Alternatively, the surface distortions of human facial skinmay be emulated with other means that do mimic any aspects of theassociated, underlying human muscular physiology and its actions; suchother means may include some, all or none of the following: a polymervideo display molded into the form of the human face, a dynamicallydistorting sheet of electroactive polymer material, video projected intoa translucent shell molded into the approximate form of the human face,and/or motion affected in the underlying mechanical frame.

[0078] Following is discussion of how the listed muscle groups may beemulated by using techniques described in this patent. Detailed studiesof the actions and geometries of the listed muscles and/or otherexpressive muscles are publicly available in the literature of medicalanatomy, anthropology, psychology, and film animation. Such literaturemay be consulted regarding geometry for emulated muscle placements,vectors of emulated muscle motions, and for the paralinguistic,narrative, and/or aesthetic meaning of individual and orchestratedmuscle actions when expressing simulated thoughts, reflexes, andemotions. Such placements, vectors, and paralinguistic decision-making,may also be customized to suit a particular embodiment, or to suit thedesires of an engineer or artist. In general, the muscles of the facethat are linear (which includes all except the Orbicularis oculi 75 andthe Orbicularis oris 82 muscle groups) may be emulated using expressionaffectors 5 and/or sliding affectors 10. The multiple complex actions ofthe ring-shaped muscles groups of the face, which include Orbicularisoculi 75 and Orbicularis oris 82, may be emulated by combinations ofexpression affectors 5, sliding affectors 10, eyelid mechanisms 11,and/or lip mechanism 8. These multiple actions may act in concert toemulate the “pursing” action of the ring-shaped muscles. Alternatively,other means may be used to achieve this affect, such as, for example, a“drawstring” approach.

[0079] The details of the above mentioned muscular emulations aredescribed below. Alternatively, any other suitable means may be used toreproduce the action of the listed muscle groups.

[0080] To emulate the action of the Frontalis 73 in the center of theforehead, a sliding affector 10 (or multiple sliding affectors) may beused, pulling the skin of the central forehead upward into positionscharacteristic of expressions that may include “surprise”, “fear”,“sorrow”, “skepticism”, “greetings”, and “curiosity”. One embodiment ofthe emulation is shown in FIG. 10, and is accordingly described indetail. Alternatively, any other suitable means may be used to reproducethe action of this muscle group.

[0081] To emulate the action of the Frontalis 73 on the outer regions ofthe forehead, a sliding affector 10 or an expression affector 5 may beused to pull the skin of the outer eyebrows upward into positionscharacteristic of expressions that may include “surprise”, “mischief”,“skepticism”, and “curiosity”. Alternatively, any other suitable meansmay be used to reproduce the action of this muscle group. To emulatethis muscle action with an expression affector 5, an anchor 34 may beembedded within the outer portion of each eyebrow. A flexible linkage 35attached to the anchor may run upward along the forehead, and then bendaround (and run through) a radiused opening present in a mechanicalframe 36. From here, the linkage 35 may extend back into the cavityinterior of the mechanical frame 36, where it may attach to an actuator33. When actuated, this linkage 35 slides around and through theradiused opening, pulling the anchor 34 and skin at an angleapproximating that of the action of Frontalis 73 muscle, so as to causethe eyebrow to arch and the skin of the outer forehead to bunch up.Alternatively, a sliding affector 10 may be embedded in the outerportion of each eyebrow, such that its slot in the mechanical frame 36extends upward under the skin of the forehead. As the sliding affector10 rides upward in the slot, the eyebrow is made to arch, and the skinabove may bunch up realistically. Alternatively, any other suitablemeans may be used to reproduce the action of this muscle group.

[0082] To emulate the action of the Corrugator 74 on the central regionsof the forehead, expression affectors 5 may be used to pull the skinthat is vertically above the central region of each eyebrow, inward andslightly down into a furrowed, bunched position that is characteristicof expressions that may include “anger”, “concentration”, “protectivebunching”, “skepticism”, “sorrow”, and “fear”. Alternatively, any othersuitable means may be used to reproduce the action of this muscle group.To emulate this muscle action using an expression affector 5, an anchor34 may be embedded in the skin that lies vertically above each eyebrow.A flexible linkage 35 may attach to the end of the anchor 34 that isnearest to the center of the forehead, and the linkage 35 may run towardthe center of the brow, angled slightly downward. Near the center of thebrow, the linkage 35 may bend around (and run through) a radiusedopening present in a mechanical frame 36, after which the linkage 35 mayextend back into the cavity interior of the mechanical frame 36, whereit may attach to an actuator 33. When actuated, this linkage 35 slidesaround and through the radiused opening, pulling the anchor 34 and skininward at an angle approximating an angle of action of Corrugator 74muscles, so as to cause the skin to bunch into a “furrow”.Alternatively, a radiused opening may be present in the body of theaforementioned sliding linkage 35 that emulates the action of Frontalis73 in the center of the forehead, so that the “scowl” may move upwardwith the motion of this linkage, so as to create enhanced realism inexpressions that may include “fear” and “sorrow”. Alternatively, anyother suitable means may be used to reproduce the action of this musclegroup.

[0083] To emulate the action of the Procerus 76 on the inner regionsunder the eyes and the center of the brow, a sliding affector 10 may beused to pull the skin that is between the eyes and in the centralforehead downward into positions that are characteristic of expressionsthat may include “squinting”, “anger”, “protective bunching”, “disgust”,“concentration”, and “skepticism”. This Procerus-emulating slidingaffector 10 may double as the one used to emulate the action of theFrontalis 73 in the center of the forehead, so that the sliding affector10 emulates Frontalis 73 when sliding upward, but emulates the Procerus76 when sliding down, as is demonstrated in FIG. 10. Alternatively, anyother suitable means may be used to reproduce the action of this musclegroup.

[0084] To emulate the part of the action of the Orbicularis oculi 75that moves the eyelids, eyelid mechanisms 11 may be used. Alternatively,any other suitable means may be used to reproduce the action of thismuscle group.

[0085] To emulate the action of the Orbicularis oculi 75 on the outerregions under the eyes, expression affectors 5 may be used to pull theskin that is under the eyes and in the cheeks upward and outward intobunched positions that are characteristic of expressions that mayinclude “squinting”, “Duchenne (true) smile”, “fear”, “protectivebunching”, “disgust”, “curiosity”, and “skepticism”. Alternatively, anyother suitable means may be used to reproduce the action of this musclegroup. To emulate this muscle action using expression affectors 5,anchors 34 may be embedded in the lower eyelids, near the eyes, and mayextend down into the cheeks. A linkage 35 may run upwards from theanchor 34, along the outside of the eye, then to bend around (and runthrough) a radiused opening present in a mechanical frame 36. Afterthis, the linkage 35 may extend back into the cavity interior of themechanical frame 36, where it may attach to an actuator 33 that may bemounted to the mechanical frame 36. Then, when actuated, this linkage 35slides around and through the radiused opening, to pull the anchor 34and skin 25 so that the skin slides upward and bunches around the cornerof the eye.

[0086] To emulate the action of the Orbicularis oculi 75 on the innerregions under the eyes, running toward the nose, expression affectors 5may be used to pull the skin that is under the eyes and in the cheeksupward and inward into bunched positions that are characteristic ofexpressions that may include “squinting”, “Duchenne (true) smile”,“protective bunching”, “disgust”, “curiosity”, and “skepticism”.Alternatively, any other suitable means may be used to reproduce theaction of this muscle group. To emulate this muscle action usingexpression affectors 5, anchors 34 may be embedded in the lower eyelids,near the eyes, and may extend down into the cheeks. A flexible linkage35 may run from an anchor 34 toward the nose, angling slightly upward,and then bend around (and run through) a radiused opening present in amechanical frame 36. From there, the linkage 35 may extend back into thecavity interior of the mechanical frame 36, where it may attach to anactuator 33 mounted to the mechanical frame 36. When actuated, thelinkage 35 may be made to slide around and through the radiused opening,pulling the anchor 34 and skin in and up so as to bunch into a “squint”.

[0087] To emulate the action of the Orbicularis oculi 75 on the portionsof the eyebrows and the skin above the eyes that extend from theapproximate center of the eyes to the outside edge of the face, slidingaffectors 10 may be used to pull the skin and outer eyebrows down andinward into bunched positions that are characteristic of expressionsthat may include “squinting”, “concentration”, “protective bunching”,“sorrow”, “curiosity”, and “skepticism”. Alternatively, any othersuitable means may be used to reproduce the action of this muscle group.A sliding affector 10 used for this affect may also double as the oneused to emulate the action of the outer regions of Frontalis 73 whenraising the eyebrow, so that the sliding affector 10 emulatesOrbicularis oculi 75 when sliding down, but emulates the Frontalis 73when sliding up.

[0088] The muscles of the eyeballs may be emulated using the eyemechanism. Alternatively, any other suitable means may be used toreproduce the action of these muscles.

[0089] To emulate the action of Levator labii superioris and/or Caninus77, which vertically pulls the outer regions of the upper lip,expression affectors 5 may be used to form the skin into positions thatare characteristic of expressions such as: some forms of smiles, “fear”,and some speech-related movements. Alternatively, any other suitablemeans may be used to reproduce the action of this muscle group. Toemulate this muscle action by using expression affector 5, anchors 34may be embedded along the sides of the mouth; these may double as theanchors used for other emulated muscles, such as, for example,Zygomaticus major 78. A flexible linkage 35 may run from the top of theanchor 34 upward parallel to the nose, to bend around (and run through)a radiused opening present in a mechanical frame 36, below the eye. Fromthere, the linkage 35 may extend back into the cavity interior of themechanical frame 36, where it may attach to an actuator 33 mounted tothe mechanical frame 36. When actuated, the linkage 35 may be made toslide through the radiused opening, pulling the anchor and skin upwardso as to bunch into a “smirk”.

[0090] To emulate the action of Zygomaticus major 78, which pulls thecorners of the lips and regions nearby on the cheeks, expressionaffectors 5 may be used to pull the skin out and upward into positionsthat are characteristic of expressions that may include “joy”, “fear”,“protective bunching”, “grief”, and some speech-related movements.Alternatively, any other suitable means may be used to reproduce theaction of this muscle group. To emulate this muscle action by usingexpression affector 5, an anchor 34 may be embedded along the corner ofthe mouth, and may extend out into the cheek. This anchor may double asthat used by other emulated muscle groups (such as, for example,Depressor labii 81), The anchor 34 may be composed in part of cloth, andmay extend some distance into the lips to reinforce the lips and thecorner of the mouth to prevent tearing. This anchor 34 may be alsopartly composed of a semi-rigid crescent of any suitable material placedalong the crease of the side of the smile, to cause the skin to foldalong this crease in a realistic manner. The anchor 34 may also bepartly composed of a cloth appendage that will produce a “dimple” in thecheek when pulled, for enhanced aesthetics and realism. A flexiblelinkage 35 may run from the cheek-side part of the anchor upward andoutward, following the cheek, to reach a point on a mechanical frame 36that is near the outer edge of the face, below the level of the eyes(geometrically, this point may correspond with the location that theZygomaticus major 78 muscle attaches to the zygomatic arch in humans).At this point, the linkage 35 bends around and runs through a radiusedopening present in the mechanical frame 36. From there, the linkage 35may extend back into the cavity interior of the mechanical frame 36,where it may attach to an actuator 33 mounted to the mechanical frame36. When actuated, the linkage 35 may be made to slide through theradiused opening, pulling the anchor and skin upward so as to bunch intoa “smile”.

[0091] To emulate the action of Zygomaticus minor 79, which elevates theupper lip and regions along the sides of the nose (particularly inexpressions of contempt), expression affectors 5 may be used to form theskin into positions that are characteristic of “sneer”, “disgust”,“anger”, “fear”, some forms of smiles, and some speech-relatedmovements. Alternatively, any other suitable means may be used toreproduce the action of this muscle group. To emulate this muscle actionby using expression affectors 5, anchors 34 may be embedded in the topof the upper lip, approximately one fourth of the way in from thecorners of the mouth. A flexible linkage 35 may run from the top of eachanchor 34 upward and outward, into the cheek, to reach a part of amechanical frame 36 toward the outer edge of the face, just below thelevel of the eyes. At this point, the linkage 35 bends around and runsthrough a radiused opening present in the mechanical frame 36. Fromthere, the linkage 35 may extend back into the cavity interior of themechanical frame 36, where it may attach to an actuator 33 mounted tothe mechanical frame 36. When actuated, the linkage 35 may be made toslide through the radiused opening, pulling the anchor 34 and skinupward so as to bunch into a “sneer”.

[0092] To emulate the action of Buccinator 80, which pulls the cornersof the lips and regions nearby on the cheeks outward, expressionaffectors 5 may be used to pull the skin out into positions that arecharacteristic of expressions that may include “joy”, “fear”,“protective bunching”, “grief”, and some speech-related movements.Alternatively, any other suitable means may be used to reproduce theaction of this muscle group. To emulate this muscle action by usingexpression affector 5, an anchor 34 may be embedded along the corner ofthe mouth, and may extend out into the cheek. The anchor 34, which maydouble as that used by other emulated muscle groups (such as, forexample, Zygomaticus major 78), may be composed in part of cloth, andmay extend some distance into the lips to reinforce the lips and thecorner of the mouth, to prevent tearing. This anchor 34 may be alsopartly composed of a semi-rigid crescent of any suitable material placedalong the crease of the smile, to cause the skin to fold along thiscrease in a realistic manner. The anchor 34 may also be partly composedof cloth appendage that will produce a “dimple” in the cheek whenpulled, for enhanced aesthetic and realism. A flexible linkage 35 mayrun from the cheek-side part of the anchor 34 horizontally outward,following the cheek, to reach a part of a mechanical frame 36 near or atthe outer edge of the face. At this point, the linkage 35 bends aroundand runs through a radiused opening present in the mechanical frame 36.From there, the linkage 35 may extend back into the cavity interior ofthe mechanical frame 36, where it may attach to an actuator 33 mountedto the mechanical frame 36. When actuated, the linkage 35 may be made toslide through the radiused opening, pulling the anchor 34 and skinoutward so as to affect an expression like that associated with the long“E” utterance.

[0093] To emulate the action of Depressor labii 81, which pulls portionsof the lower lip downward, expression affectors 5 may be used to pullthe skin out into positions that are characteristic of expressions thatmay include “fear”, “critical thinking”, “grief”, and somespeech-related movements. Alternatively, any other suitable means may beused to reproduce the action of this muscle group, such as, for example,lip mechanism 8. To emulate this muscle action by using expressionaffector 5, an anchor 34 may be embedded in the lower lip, approximatelyone fourth of the way in from the corners of the mouth. A flexiblelinkage 35 may run downward from the lower part of the anchor 34, toreach a part of a mechanical frame 36 at or near the jawline. At thispoint, the linkage 35 may bend around and run through a radiused openingpresent in the mechanical frame 36. From there, the linkage 35 mayextend back into the cavity interior of the mechanical frame 36, whereit may attach to an actuator 33 mounted to the mechanical frame 36. Whenactuated, the linkage 35 may be made to slide through the radiusedopening, to pull the anchor 34 and skin downward so as to affect anexpression like a frown.

[0094] Lip mechanism 8 may be used to emulate Orbicularis oris 82 in itsaction to raise, lower and/or push out parts of the lips, actions thatare characteristic of expressions that may include “fear”, “joy”,“disgust”, “sorrow”, and speech-related movements. Alternatively, anyother suitable means may be used to reproduce these actions of thismuscle group, such as, for example, expression affector 5. Optionally,to increase the expressivity of the upper lip, multiple lip mechanisms 8may impart motion to different discrete sections along the length of theupper lip. Lip mechanisms, in their ability to impart force on thereturn stroke, may pull the lips as well, and in so doing, may alsoserve to emulate the action of muscles that move opposite to Orbicularisoris 82, such as, for example, Mentalis 83, or Depressor labii 81.

[0095] To emulate part of Orbicularis oris' 82 action that purses thelips, expression affector 5 may be used to move the corners of the lipin toward the center of the lips, and/or forward and out towards thefront of the face. These actions are characteristic of expressions thatmay include “surprise”, some expressions of anger and suspicion, andmany speech-related movements. Alternatively, any other suitable meansmay be used to reproduce this action of this muscle group. If expressionaffector 5 is used to emulate this action, an anchor 34 may be embeddedin the skin at the corner of the mouth. This anchor 34 may also,simultaneously, serve as an anchor 34 for other emulated muscle groups(such as, for example, Zygomaticus major 78). A substantially rigidlinkage 35 may attach to the anchor 34, and then extend back through anopening in a mechanical frame 36 to attach to an actuator 33. Whenactuated, this linkage 35 may push on the anchor 34, which distributesforce into the skin 25, causing the skin to project forward. This actionmay or may not be produced by the same linkage and actuator 33 that areused to emulate the action of Zygomaticus major 78, such that when theactuator 33 pulls on the linkage, the resulting motion emulates theaction of Zygomaticus major 78, but when the actuator 33 pushes thelinkage 35 outward, the resulting motion emulates the action ofOrbicularis oris 82. Optionally, a drawstring of any suitable materialmay be embedded in the lips so that when pulled, it may decrease thediameter of the mouth opening to affect the appearance of pursing lips,emulating this action of the Orbicularis oris 82. Such a drawstring mayalso be given slack to allow the diameter of the mouth to increase forsome expressions, such as, for example, a wide-open smile.

[0096] To emulate the action of Triangularis 84, which pulls the cornersof the lips and nearby regions downward, expression affectors 5 may beused to pull the skin out into positions that are characteristic ofexpressions that may include “fear”, “critical thinking”, “grief”, andsome speech-related movements. Alternatively, any other suitable meansmay be used to reproduce the action of this muscle group. To emulatethis muscle action by using expression affector 5, an anchor 34 may beembedded along the corner of the mouth, and may extend out into thecheek. The anchor 34, which may simultaneously serve as an anchor forother emulated muscle groups (such as, for example, Zygomaticus major78), may be composed in part of cloth, and may extend some distance intothe lips to reinforce the lips and the corner of the mouth, to preventtearing. A flexible linkage 35 may run downward from the lower part ofthe anchor 34, to reach a part of a mechanical frame 36 at or near thejawline. At this point, the linkage 35 may bend around and run through aradiused opening present in the mechanical frame 36. From there, thelinkage 35 may extend back into the cavity interior of the mechanicalframe 36, where it may attach to an actuator 33 mounted to themechanical frame 36. When actuated, the linkage 35 may be made to slidethrough the radiused opening, to pull the anchor 34 and skin downward soas to affect an expression like a frown.

[0097] To emulate the action of Nasalis 85, which pulls the regionsalong the sides of the nose into wrinkles, expression affectors 5 may beused to form the skin into positions that are characteristic ofexpressions that may include “sneering”, “disgust”, some forms of“anger” and “fear”, and some forms of smiles and speech-relatedmovements. Alternatively, any other suitable means may be used toreproduce the action of this muscle group. To emulate this muscle actionby using expression affector 5, anchors 34 may be embedded along thesides of the nose. A flexible linkage 35 may run from the top of theanchor 34 upward on the bridge of the nose, to then bend around (and runthrough) a radiused opening present in a mechanical frame 36. Fromthere, the linkage 35 may extend back into the cavity interior of themechanical frame 36, where it may attach to an actuator 33 mounted tothe mechanical frame 36. When actuated, the linkage 35 may be made toslide through the radiused opening, pulling the anchor 34 and skinupward so as to bunch into a “sneer”.

[0098] To emulate the actions of the muscles of the jaw 87, an actuator33 may be used to rotate the artificial jaw of the HER.

[0099] Following is discussion of how emulated muscle groups may be usedin orchestration so as to represent 6 basic emotional expressions:“joy”, “sorrow”, “surprise”, “anger”, “fear”, and “disgust”. The actionsof the emulated muscles—in regards to specific geometries of emulatedmuscle placement, the vectors of emulated muscle motion, and theparalinguistics of individual and orchestrated muscle actions—may alsobe customized to suit a given embodiment or to suit the desire of anengineer or artist. In the process of such customization, one may or maynot refer to public literature of animation, anthropology, psychology,anatomy and physiology for design inspiration. The 6 simulated facialexpressions in the following examples are chosen because they areidentified by anthropologists as being universal, transcultural, and inregular use for human nonverbal communications. Alternatively, thedescribed system of emulated facial muscles may be used to express othersimulated cognitive states, intentions, emotions, thoughts, desires,affections, affectations, and/or simulated speech motions. In the given6 examples, the extremity of action of each emulated muscle group may bemodulated as desired so as to create subtle variation in the meanings ofeach expression. Also, some of the 6 basic expressions may be blendedand actuated simultaneously so as to create complex, or mixed,expressions, such as, for example: joyous-surprise, or disgusted-anger.For, the purpose of discussion, all emulated muscle actions may bepresumed to be actuated to their non-destructive extremes, but inpractice these actions may be modulated as desired.

[0100] The expression of “joy” may be achieved by actuating some or allof the following emulated muscle actions: Zygomaticus major 78 (pullingthe lip corners outward and up), Zygomaticus minor 79 (raising the upperlip), Buccinator 80 (pulling the lips wide), Orbicularis oris 82(raising the upper lip), Depressor labii 81 (drawing the lower lip openand widening it), and Mentalis 83 (drawing the lower lip and chindownward). Optionally, to achieve a “truthful” or “Duchenne” smile, onemay additionally actuate the emulated Orbicularis oculi 75 in the regionunder the eyes and at the edge of the eyes, and/or eyelid mechanism 11to cause the lower eyelids to raise, so as to affect a “squint” of theeyes.

[0101] The expression of “sorrow” may be achieved by actuating some orall of the following emulated muscle actions: Triangularis 84 (frowningthe corners of the mouth), Frontalis 73 (raising the center region ofthe forehead/brow), Orbicularis oris 82 (pushing the lower lip andprojecting it forward), Mentalis 83 (pushing the lower lip upward),Orbicularis oculi 75 (pulling the arch of the eyebrows down and inward),and/or Corrugator 74 (furrowing the brow).

[0102] The expression of “surprise” may be achieved by actuating some orall of the following emulated muscle actions: Frontalis 73 (pulling allthe forehead and eyebrows upward), eyelid mechanisms 11 (affecting “wideopen” eyes), Depressor labii 81 (pulling the lower lip down and drawingit outward), Platysma 86 (opening the jaw), and/or Mentalis 83 (pullingthe lower lip down).

[0103] The expression of “fear” may be achieved by actuating some or allof the following emulated muscle actions: Zygomaticus major 78,Zygomaticus minor 79, Triangularis 84, Corrugator 74, Depressor labii81, Frontalis 73, Procerus 76, Nasalis 85, Levator labii superioris 77,and/or the Mentalis 83.

[0104] The expression of “anger” may be achieved by actuating some orall of the following emulated muscle actions: Procerus 76 (dropping thebrow into a scowl), Corrugator 74 (furrowing the brow), Zygomaticusminor 79 (sneering the upper lip), Nasalis 85 (sneering the nose),Triangularis 84 (frown), Depressor labii 81 (dropping inner section ofthe lower lip), Orbicularis oris 82 (dropping the arch of the eyebrow,and scrunching the skin below the eye into a squint), eyelids (uppereyelids raise to open the eyes wider), and/or the Orbicularis oris 82(pursing the lips, and raising the upper lip)

[0105] The expression of “disgust” may be achieved by actuating some orall of the following emulated muscle actions: Procerus 76, Zygomaticusminor 79, Nasalis 85, Triangularis 84, Orbicularis oris 82, Orbicularisoculi 75 (to scrunch the skin below the eyes into a squint), and/or theMentalis 83.

[0106]FIG. 13 is a block diagram illustrating one embodiment of acontrol system configuration that may be used to emulate at least somehuman communication in an HER. In the depicted embodiment, video sensor60 and audio sensor 67 in a HED 12 relay video and audio data into acomputational system 88. One set of software and/or embedded hardwareprocesses running in the computational space, called Automatic SpeechRecognition (ASR) 89, processes the audio data to detect speech andextracts words and low-level linguistic meaning. Any suitable softwaremay be used to accomplish ASR, including various available systems, suchas, for example, Sphinx from Carnegie Mellon University or IBM'sViavoice. Another set of software and/or embedded hardware processescalled Computer Vision 90 may perform any of various visual perceptiontasks using the video data. Such visual perception tasks may include thedetection and tracking of the faces of people, using any suitablesoftware, which may include various available software, such as, forexample, SRI's People Tracker or Eyematic's FaceTracker. Other visualperception processes may also include the tracking of facial features orexpressions on the faces of people and any suitable software may be usedfor this task, such as, for example, Eyematic's FaceStation. Decisionprocesses 91 may be used to determine appropriate communicativeresponses and other behaviors and any suitable software may be used forthis purpose such as, for example, any of the public or commerciallyavailable “chatbot” software, like “eliza” and “Alice.” Alternatively,those skilled in the art may create software specific to theapplications described above to implement the claimed invention. Otherprocesses called special programs 92 may be any off-the-shelf or customsoftware that may be bundled with the HER, that include functions thatenhance the HER for specific applications are not included in standardsoftware, such as, for example: software that monitors vital functionsin a patient, software that detects presence of chemical danger, and/orsoftware that set off fire alarms or burglar alarms. Other processescalled specific programs 93 may be used to enhance the functionality ofthe system in other ways. These specific programs 93 may be anyoff-the-shelf software that may be accessed by the HER, wherein the HERbecomes a layer through which a user interacts with the specificprograms; such software may include any, all, or none of the following:spreadsheet software, web-browsing software, email programs, wordprocessing software, electronic books, expert systems, ArtificialIntelligence software, mathematical software, geo-positioning software,and/or game software. Data storage 94 may be used to affect a long-termmemory, build knowledge, and to store software. Speech responses may besynthesized by any suitable speech synthesis software 95 such as, forexample, the open-source software “Festival”, to be made audible usingan audio transducer 70. Any suitable motor controller 96 may be used toeffectuate non-verbal expressions in the HED 12. Nonverbal responses mayoperate independent of speech (in one embodiment, by direct mimicry), orin synchrony with the content of the synthesized speech.

[0107] The following section discusses flow of data through oneembodiment of an HER control system. The cameras in the HED's eyes maysend video signals into the computer vision module 90, which locates ahuman face in the video stream. The data regarding the position of thisperson may be sent on to decision processes 91, which may decide toreact to the person in the video signal, based on variables that governthe HER's attention. If the HER decides to react to the perceivedperson, data will be sent to motor controller 96 regarding theanticipated position of the person. Motor control signals may travel toactuators 33 in the HED that control eye position, and optionally, tomotors that govern head-pose, to make eye contact with the perceivedperson. Decision processes 91 may also send a signal to motor controller96 to affect a smile of greeting; motor controller 96 may generate andsend appropriate the signals to the actuators responsible for a joyfulsmile. Decision processes 91 may also send the signal to speechsynthesis 95 to utter a greeting. Speech synthesis 95 may generate thecorresponding speech greeting signal, and send this signal to an audiotransducer 70 in the HED, to make the signal audible to a person. Uponthese greeting gestures, the person may return greeting, and thenrequest if any new email messages have been sent for the person. Themicrophones in the HED may detect the speech data, transmit the data toASR 89, which may decipher the words, and may decipher some linguisticcontent. The resulting language analysis will be relayed to decisionprocesses, which may further decipher the linguistic content, tounderstand the person's request. Decision processes may summon specialprograms 92 dedicated to personal assistant functions, to translate therequest for email into appropriate web-browsing functions. Specialprograms 92 may then send signals to specific programs 93, which openthe appropriate servers and URLs on the Internet, to open the person'semail account. Special programs 92 then decipher the hypertext of themessages to determine what email messages are new, and returns this datato decision processes 91, which puts the information into naturallanguage, which is relayed to speech synthesis 95, which then generatesspeech signals that are relayed to the audio transducer 70 in the HED.The transmission of this audible signal completes the control loop.During the described transactions, all data may be relayed from decisionprocesses, special programs, and specific programs, to data storage 94,where it may be recorded for future reference as needed. Data storage 94may also serve to store software that is not actively used, to besummoned into active duty as needed, to conserve processing resources.

[0108]FIG. 14 is a block diagram illustrating an embodiment of a HumanEmulation Robot being used to access, navigate, or modify theinformation present on a computer network. This embodiment of a HumanEmulation Robot 1 comprises an expressive, human-appearing mechanism 12,and a computer system to generate interactive, human-like conversationin said mechanism 13, wherein said computer connects to a computernetwork 97 in such a way that said computer can access data 98 presentin other computers 99. This data is then translated into emulatednatural conversation for the human user 100. In one embodiment, a user100 may ask the Human Emulation Robot 1 if one has any new email, whichmay cause the Human Emulation Robot 1 to access the user's email via theInternet 101. The Human Emulation Robot 1 may then announce in naturallanguage, “Yes, you have new email from your grandmother”. To which theuser 100 may then inquire, “Well, what does she say?”, which may causethe Human Emulation Robot 1 to recite the message to the user 100. Inanother embodiment, the Human Emulation Robot 1 may find an encyclopediaarticle from the World Wide Web 102 at the spoken behest of a user 100,and recite the article in synthesized spoken language. Alternatively,the HER may simply help locate the article and the article may bedisplayed on a screen 103 for the user to read. The HER could be used asa computer interface in any number of ways. Speech recognition iscurrently used in various computer interface applications and the HERcould be used in all of these ways. The HER could serve as a virtualsecretary and take dictation of a letter that ends up in aword-processing program. Rather than simply sounding a tone when anerror occurs, however, the HER could say that an error has occurred andexplain what the error is, thus enhancing the user's experience andmaking the computer easier to use. The HER could serve as a “desktop”assistant which interacts with the user using natural language in orderto navigate, access, open, and modify applications, documents, andsystem software on a computer. In this case, the HER may verballydescribe file names, actions that may be taken, and interpret contentsof files. Alternatively, the HER may simply help the user to locatefiles and then display the files on a screen. The HER could serve as avirtual personal assistant, by helping to organize the files on acomputer, in response to natural dialogue and visual gestures from theuser. The HER could furthermore serve as a virtual personal assistant,by managing one's schedule, taking memo notes, and reminding one ofimpending appointments, events, and important dates. In expanded rolesas a virtual personal assistant, the HER could access variousnatural-language ticketing agents, sales agents, search-bots, and othernatural language agents that are increasingly common on the World WideWeb and elsewhere in commerce, allowing the user to book tickets, searchfor answers to questions, and so on. The natural language of theseagents could be interpreted and presented by the HER so as to maintainthe natural flow of the user's dialogue with the HER. In so doing, theHER may provide a face for these outside, automated agents, and theseagents may expand the intelligent capabilities of the HER. The HER couldserve as an alternative display for various computer-generatedcharacters (the HER being three-dimensional and physically embodied asopposed to “virtual” imagery on a display screen) for animated agents,“chatterbots”, or avatars. In one such embodiment, the HER could serveas an interface to a video game, such that the HER becomes controlled bya video-game character. In essence, the character possesses the body ofthe HER, and in so doing, comes to exist outside of the game, in ourphysically-embodied world, thus enhancing the experience of thevideo-game user, and opening many new possibilities in video-gamedesign.

[0109]FIG. 15 is a block diagram illustrating a method for producing anartificial skin material, comprising: building a three dimensionalmatrix 103 made of removable material, perfusing the matrix withelastomer 104 that is not as easily removable as said removablematerial, and removing the matrix to leave behind a web of elastomer 105that is permeated by at least one network of voids 106, wherein the webof elastomer is flexibly compressible while maintaining at least someelastic characteristics. Various details and options related to thismethod were discussed above.

[0110] Although the present invention has been described in detail, itshould be understood that various changes, substitutions and alterationscan be made hereto without departing from the sphere and scope of theinvention as defined by the appended claims.

[0111] To aid the patent office, and any readers of any patent issued onthis application in interpreting the claims appended hereto, applicantswish to note that they do not intend any of the appended claims toinvoke paragraph 6 of 35 U.S.C. § 112 as it exists on the date of filinghereof unless “means for” or “step for” are used in the particularclaim.

I claim:
 1. A robot system comprising: (a) a flexible artificial skinoperable to be mechanically flexed under the control of a computationalsystem, (b) a first set of software instructions operable to receive andprocess input images to determine whether at least one human likely ispresent, and (c) a second set of software instructions operable todetermine a response to a determined likely human presence, whereby thecomputational system shall output signals corresponding to the response,such that, in at least some instances, the output signals cause thecontrolled flexing of the artificial skin.
 2. The system of claim 1,wherein the artificial skin comprises a foamed elastomer.
 3. The systemof claim 1, wherein the artificial skin comprises a porous elastomer. 4.The system of claim 1, wherein at least some portion of the artificialskin resembles at least some portion of a human face.
 5. The system ofclaim 1, further comprising actuators and software operable to cause theactuators in combination with at least the artificial skin to emulate atleast some human nonverbal communications.
 6. The system of claim 1,further comprising a screen-based display.
 7. The system of claim 1,further comprising: (a) at least one video camera operable capture theinput images which are input to the computational system in digital oranalog form, (b) image processing software in the first set of softwareinstructions operable to determine whether at least one human facelikely is present in the input images, and (c) decision software in thesecond set of software instructions operable to determine an output thatcauses the artificial skin to emulate a human expression.
 8. The systemof claim 1, further comprising: (a) an artificial eye in the face of therobot, and (b) a video camera at least partially coupled to theartificial eye, and operable to capture the input images which are inputto the computational system in digital or analog form.
 9. The system ofclaim 1, further comprising: (a) at least one audio transducer operableto capture input sounds which are input to the computational system indigital or analog form, (b) a third set of software instructionsoperable to receive and process the input sounds and indicate soundpatterns in the data, and (c) decision software in the third set ofsoftware instructions operable to determine audio pattern output signalsthat causes sound patterns to be output through a speaker associatedwith the robot.
 10. The system of claim 9, further comprising: (a)wherein the third set of software instructions is further operable toperform automatic speech recognition, (b) wherein the third set ofsoftware instructions is further operable to determine the word outputsignals in response to words recognized during automatic speechrecognition, (c) decision software in the third set of softwareinstructions operable to determine word output signals that causes wordsto be output through a speaker associated with the robot, and (d)wherein the output signals are further operable to control at least somemotions of the artificial skin in synchrony with the word outputsignals.
 11. The system of claim 1, further comprising a mechanicalsystem operable to create controlled motion in the flexible artificialskin, comprising: (a) an anchor fixed to the artificial skin, and (b) amember operable to actuate the anchor such that the anchor distributesforce through the artificial skin to control motion of the artificialskin.
 12. The system of claim 1, further comprising a mechanical systemoperable to create controlled motion in the flexible artificial skin,comprising: (a) a compound series of linkages connected at jointsoperable to act as compound levers, and wherein at least some of thelinkages are affixed to the artificial skin, and (b) a plurality ofactuators operable to impart motion to a plurality of the linkages tocause compound motion of the artificial skin.
 13. The system of claim 1wherein the robot functions as an interface between at least one humanand a computer network.
 14. The system of claim 1 wherein the robotfunctions as an automated assistant to humans.
 15. The system of claim 1wherein the robot functions as an advertising medium.
 16. The system ofclaim 1 wherein the robot functions as a therapy tool.
 17. The system ofclaim 1 wherein the robot functions as a tool for education.
 18. Thesystem of claim 1 wherein the robot functions as a toy.
 19. The systemof claim 1 wherein the robot functions as an entertainment medium. 20.The system of claim 1 wherein the robot functions as a security orpolice tool.
 21. A method for interaction between a human and a robot,comprising: (a) electronically determining whether a human likely ispresent, (b) electronically determining an output response to adetermination that a human is likely present, and (c) controllingmechanical flexing of artificial skin associated with the robot inresponse to the determined output response.
 22. A method as described inclaim 21, wherein the artificial skin comprises a foamed elastomer. 23.The method of claim 21, wherein the artificial skin comprises a porouselastomer.
 24. The system of claim 21, wherein at least some portion ofthe artificial skin resembles at least some portion of a human face. 25.The method of claim 21, further comprising using automatic speechrecognition to help to determine the output response.
 26. The method ofclaim 21, wherein the artificial skin comprises a portion of the robotthat resembles a human face.
 27. The method of claim 21, furthercomprising using the robot as an advertising medium.
 28. The system ofclaim 21, further comprising using the robot as a tool for education.29. The system of claim 21, further comprising using the robot as a toy.30. The system of claim 21, further comprising using the robot as anentertainment medium.
 31. The system of claim 21 wherein the robotfunctions as a security or police tool.
 32. An artificial skin materialcomprising a mixture of freshly catalyzed elastomer that is still liquidand not yet fully cured and a freshly catalyzed foaming elastomer thatis also still liquid and curing, such that the resulting cured materialalternates between discrete, microscopic sections of nearly pureelastomer and discrete sections of foaming elastomer, whereby thematerial is easily compressed because it is permeated by gas-bubblevoids, yet remains elastic because the material is interfused by areasof nearly pure elastomer.
 33. A method for making an artificial skinmaterial, comprising: (a) building a three dimensional matrix made ofremovable material, (b) perfusing the matrix with elastomer that is notas easily removable as the removable material, and (c) removing thematrix to leave behind a web of elastomer that is permeated by at leastone network of voids, wherein the web of elastomer is flexiblycompressible while maintaining at least some elastic characteristics.34. The method of claim 33, further comprising depositing the matrix ofremovable material with an electronically controlled three dimensionaldepositional printer, so as to control the structural form of the matrixof removable material, such that when the matrix is perfused withelastomer, the resulting web of elastomer adopts the negative imprint ofthe controlled structural form of the matrix, thus determining thestructural form of the porous voids in the web of elastomer.
 35. Themethod of claim 33, further comprising assembling the matrix by shakingloose particles of the removable material.
 36. The method of claim 33,further comprising assembling the matrix from diminishingly scaledparticles such that smaller particles fill interstices left by largerparticles.
 37. The method of claim 33, further comprising structuringpores such that the web of elastomer mechanically deforms in controlledways under strain.
 38. The method of claim 33, further comprisingassembling the matrix such that stress on intermittent thicker areas inthe web of elastomer is relieved when the web of elastomer is understress.
 39. Artificial skin material made by: (a) building a threedimensional matrix made of removable material, (b) perfusing the matrixwith elastomer that is not as easily removable as the removablematerial, and c) removing the matrix to leave behind a web of elastomerthat is permeated by at least one network of voids, wherein the web ofelastomer is flexibly compressible while maintaining at least someelastic characteristics.